DOCUMENT RESUME 



ED 409 184 



SE 060 345 



PUB TYPE 
EDRS PRICE 
DESCRIPTORS 



AUTHOR 

TITLE 



PUB DATE 
NOTE 



IDENTIFIERS 



O'Neill, D. Kevin 

Bluffing Their Way into Science: Analyzing Students' 
Appropriation of the Research Article Genre. 

Apr 97 

22p . ; Paper presented at the Annual Meeting of the American 
Educational Research Association (Chicago, IL, April, 1997) . 
Reports - Research (143) -- Speeches/Meeting Papers (150) 

MF01/PC01 Plus Postage. 

^Communication (Thought Transfer) ; *Content Area Writing; 
^Discourse Modes; Educational Innovation; High School 
Students; Intermediate Grades; Literary Devices; Literary 
Genres; Middle Schools; Problem Solving; ^Research Papers 
(Students) ; ^Science Education; Science Process Skills; 
Secondary Education; Writing (Composition) 

Middle School Students 



ABSTRACT 



This paper reports on research in the analysis of high 



school and middle school students' appropriation of the Research Article 
genre in science classes. The appropriation of this rhetorical form is 
proposed as a measure of students' understanding of adult argumentative 
practice in science and the effectiveness of a learning environment in 
supporting the development of this understanding. An important part of this 
research has been the development of a coding scheme to enable the comparison 
of genre appropriation patterns across a large number of texts from a variety 
of school and curricular settings. The coding scheme produces a series of 
numerical scores to indicate such things as students' fulfillment of the 
standard rhetorical moves of scientific research articles, the written 
personas that students project, and the ways in which they use sources and 
authorities to support argument. Because the analysis of genre appropriation 
is a relatively non- invasive way of conducting research (when compared to 
survey instruments, for example) , this method can provide a useful tool for 
reformers to compare outcomes from iterations or conditions of curricular 
experiments aimed at developing students' understanding of adult persuasive 
practices in the sciences. Contains 16 references. (Author/NB) 
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Abstract 



This paper will report on research in the analysis of high school and middle school students' appropriation of the 
Research Article genre in science classes. The appropriation of this rhetorical form is proposed as a measure of 
students’ understanding of adult argumentative practice in science and the effectiveness of a learning 
environment in supporting the development of this understanding. 

An important part of this research has been the development of a coding scheme to enable the comparison of 
genre appropriation patterns across a large number of texts from a variety of school and curricular settings. The 
coding scheme produces a series of numerical scores to indicate students’ fulfillment of the standard rhetorical 
moves of scientific research articles, the written personas that students project, the ways in which they use 
sources and authorities to support argument, and so on. 

Because the analysis of genre appropriation is a relatively non invasive way of conducting research (when 
compared to survey instruments, for example), I believe this method can provide a useful tool for reformers to 
compare outcomes from iterations or conditions of curricular experiments aimed at developing students' 
understanding of adult persuasive practices in the sciences. 

Introduction: Textual genres in communities of practice 

Implicit in the idea of a community are elements of stability and dynamism; continuity and 
change; shared interests and ongoing contention of interests. As a result of these mixtures, 
participants in communities of practice may find themselves arguing over similar issues again 
and again. Genre theorists (Miller 1984, Bazerman 1988, Swales 1990) have observed that over 
time, these recurring rhetorical problems may give rise to customary forms of communication. 
For example, because academic researchers repeatedly find themselves applying for jobs and 
funding, they have developed the genres of the curriculum vita, the letter of recommendation and 
the grant proposal, which are easily recognizable to initiates. Today, these' genres have such 
well-wrought expectations surrounding them that they have come to form a constraint on 
acceptable practice in these communities. To write a curriculum vita in an uncustomary way is, 
therefore, to risk unemployment. 

Genres are an interesting and important phenomenon because they embody the norms and 
values of the communities that produce and reproduce them. They give newcomers to a 
community a sense of its public priorities: its collective sense of what is relevant and important 
to its ambitions. From an individual's perspective, genres are also useful because they structure 
the rhetorical problems associated with contributing to a community's discourse: 

A genre is a socially recognized, repeated strategy for achieving similar goals in situations socially perceived as 
being similar. A genre provides a writer with a way of formulating responses in certain circumstances and a 
reader a way of recognizing the kind of message being transmitted.... Thus the formal features that are shared by 
the corpus of texts in a genre and by which we usually recognize a text’s inclusion in a genre, are the 
linguistic/symbolic solution to a problem in social interaction. (Bazerman 1988, p. 62) 

In writing as elsewhere, a well-structured problem is easier to deal with than an ill-structured 
one. Thus, as Bazerman points out above, a genre does some of the writer's work for her. In this 
way, genres represent a brand of distributed intelligence (Pea 1993) which can itself be 
considered one of the accomplishments of a community of practice. Genres are truly community 
property. 

Goals of this paper 

In recent years there has been an explosion of scholarly interest in the nature and variety of 
textual genres, and what they reveal about the discourse communities in which they are produced 
and reproduced. Much of this interest stems from a belief that a deeper understanding of genres 
would assist instruction (e.g. Cope and Kalantzis 1993, Freedman 1993, Bazerman 1988). I 
share this belief. In this paper I will present results from my use of the genre construct to 
evaluate an innovation in K-12 science teaching. At the heart of this effort has been the 
production of a practical instrument for analyzing students' appropriation of the scientific 
research article genre to the presentation of their work in science classes. This instrument 
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synthesizes my own observations of students' written work at the high school and middle school 
level with scholarship about the research article genre from applied linguistics (Swales 1985) and 
the sociology of science (Bazerman 1988, Myers 1990). Before I discuss the development and 
use of this instrument, however, I would like to spend a few pages framing the research issues 
that it was designed to explore. 

Appropriating scientific genres in the classroom 

As genre theory would suggest and as researchers have previously illustrated, school and work 
settings develop their own unique genres of oral and written discourse (Berkenkotter and Huckin 
1995, Yates and Orlikowski 1992). To deal with their recurring rhetorical problems, for 
instance, schools have developed the genres of the report card, the teacher's disciplinary note to 
parents, and the 5-paragraph theme. This is natural enough, given that the school is a community 
in its own right and necessarily develops its own unique norms. Nevertheless, the school's 
mission to help students understand, and possibly participate in communities of practice beyond 
the school militates against this separation of genres. Helping students learn to write in the 
customary genres of adult work settings — from business memos in typing class to lab reports 
and research articles in science classes — has been a developing part of practice in schools since 
the 1900s (Russell 1991). In this paper I will explore one area of teaching and learning practice 
in which school genres and work genres mindfully intersect: K-12 students' attempts to 
appropriate authentic genres of professional science. 

Buffing their way into science 

Under common K-12 teaching conditions (the lab report, the science fair project), student 
writer/researchers are called upon to do some fairly extraordinary impersonations. Despite 
knowing little about the histories of the disciplines, not feeling invested in their discourses or 
involved in their stakes, they must attempt to produce pieces of writing that imitate those of 
initiates: 

The student has to appropriate (or be appropriated by) a specialized discourse, and he has to do this as though he 
were easily and comfortably one with his audience, as though he were a member of the academy or an historian 
or an anthropologist or an economist; ...He must learn to speak our language. Or he must dare to speak it or to 
carry off the bluff, since speaking and writing will most certainly be required long before the skill is “learned.” 
(Bartholomae 1985, p. 135) 

Despite these complications, many of the students and teachers whom I have interviewed in my 
research seem to feel that imitating professional scientific writing in the classroom makes a lot of 
sense. The kinds of sense it makes to them may, however, vary considerably. To give you some 
impression of this. Figure 1 summarizes the responses of 12 high school students in a project- 
based science class to the question, "Why does your teacher have you write project reports in the 
particular format that he does?" I posed this question in an open-ended fashion during a series of 
focus groups. The responses were categorized post hoc. 
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It is important to mention that the activity in this particular class, which I will discuss at greater 
length later, was largely structured around the production of a scientific research paper in a 
quasi-professional format. Thus, there may have been considerably stronger integration of the 
writing and research tasks in this setting than in many others. Some indication of this is provided 
by the fact that an equal number of students cited guidance in their research as a reason for 
writing in an authentic genre as mentioned guidance in their writing. This suggests that the 
students saw and appreciated the interconnectedness of these two processes in the classroom 
environment. 

There is not, however, a clear consensus among these students about their teacher's reasons 
for having them write in this genre. While perhaps of less salience to them than guidance in 
research and writing, the 12 students mentioned a variety of other reasons for their teacher's 
insistence on an authentically-formatted report. Five students suggested that part of the purpose 
of writing in an authentic scientific genre was to help them accurately play the role of scientists 
in the class. In this conception, the genre is an important prop in a collective game of academic 
make-believe. What may be most surprising is that so few students felt the format of the report 
was chosen for pragmatic reasons such as helping the teacher to grade the paper (by making the 
absence of valued elements more obvious), or helping him manage the class by communicating 
expectations more clearly. 

These data aside, one may ask whether it is really necessary for students to imitate genres of 
professional science in order to appreciate or understand science. Does it even help? The 
answer to both questions can be yes or no, I believe, depending on whether you are personally 
more concerned with science “content” (past findings and theories) or science “process”. To 
show you how I arrive at my own answer. I'll ask to you to imagine how you might have students 
write up a chemistry lab that bears no similarity to the way that a professional scientist would 
write it. Take your time. Likewise, can you to think of a way to have students report on a 
lengthy research project in science that looks nothing like the way that professional scientists do 
it? The idea is, I think, absurd. 

I would argue that imitation of professional practice is implicit in the very notion of teaching 
science. Even if you are not convinced of this, I hope to make it clear from the examples I will 
explore below that for at least a century, educators have found it difficult to avoid the imitative 
impulse. They continue to find it difficult. The question for education researchers is what 
purposes this imitation is intended to serve, and how well it actually serves them. 
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How K-12 students encounter scientific genres 

Students come under a variety of influences in the acquisition of scientific genre knowledge. 

With regard to traditional resources, these influences include textbooks, encyclopedias, the style 
guides that may be prepared by a school or school board, and handouts or class displays used by 
teachers. Most important, however, are the pieces of scientific writing that students are asked to 
produce themselves. These include the presentation boards that students involved in science fairs 
are commonly asked to prepare (which usually have a strict prescribed format), and the lab 
reports that they may be asked to prepare after a hands-on lab. 

Sutton (1989) provides a comparison of guidelines provided to students for writing up labs as 
far back as 1898: 

Looking into the origins of this pattern of writing. ..it is very interesting to see the variation in the flexibility 
allowed, and in how much emphasis is placed on the preliminary statement of ideas. One extreme may be 
represented by C.B. Owen of Stowe School in his Methods for Science Masters (1956). He offered the 
mnemonic: High Powered Motors Often Crash, to trigger recall of the need for Heading, Picture, Method, 
Observations and Conclusion. ...MacNair (1904) suggested: ‘The Object Aimed At’, ‘What Was Done’, ‘What 
Was Seen’ and ‘What the Result Proved’. ...A.G. Hughes (1933) advocated the headings ‘Purpose’, ‘Apparatus’, 
‘Observation’, ‘Inference’.... He stressed the importance of discussion before practical work to clarify its 
purpose.... (Sutton 1989, p. 139) 

Clearly, each of these sets of guidelines reflects different ideas about the purposes behind 
students' imitation of adult scientific practice. In the course of his review, Sutton characterizes 
two general classes of guidelines among those he observed: those that depict science as a 
regimen of careful recording (‘Science as “Describing What Happens’”), and those that depict 
science as a regimen of withholding judgment until all the data are in (‘Science as “Data First 
and Theory Later’”). Both classes of guidelines send particular messages to students about the 
nature of scientific practice, as indeed such guidelines do in the world of adult professional 
practice (Bazerman 1988, Chap. 9 discusses the messages of the APA guidelines). In effect, 
these guidelines emphasize different sets of rhetorical problems for authors and identify different 
genres through which solutions to those problems can be developed. 

Lest there be any doubt, educators still develop and use a variety of guidelines for science 
writing. The poster shown in Figure 2 was observed in a middle school science classroom in 
1996. In it you will notice some similarities to the guidelines Sutton describes above, though I 
might argue that this poster presents a more inclusive view of scientific practice. Unlike most of 
Sutton's examples, it does not focus narrowly on the act of observation, and actually encourages 
students to generate hypotheses before the outcome of an experiment is known. However, it has 
its own flaws. Through its illustrations, the poster actually mystifies the process of hypothesis- 
generation (which it pictures as a child gazing into a crystal ball), and encourages the idea that 
“research” is something bookish, done in the library alone. By placing Research in order after 
Purpose, it also obscures the possibility that the Purpose of an investigation might emerge out of 
reading something (such as a peer’s research). In itself this does considerable violence to the 
idea of a scientific community and obscures the relationship between genres of reporting and the 
conduct of research. Finally, like Sutton's examples, this poster continues to give preferential 
place to experimental protocol in the development of scientific knowledge. In fact, a great deal 
of scientific practice does not involve much laboratory experimentation (for instance Astronomy, 
Atmospheric Science, Botany, or Ecology). 
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Figure 2 “Scientific Method” poster observed in a middle 
school science classroom 



The captions to the sections read: 

“Purpose: What do you want to 
learn?” 

“Research: Find out what other people 
know...” 

“Hypothesis: Predict the answer to the 
problem” 

“Experiment: Design a test to confirm 
or disprove your hypothesis” 

“Analysis: Record what happened 
during the experiment” 

“Conclusion: Was your hypothesis 
correct?” 



I do not wish to be harsh in my assessment of these guidelines for scientific writing. If we accept 
that school science needs to imitate adult science in some respects, we must ask which ones ' ? 

Each of the representations of scientific research and reporting described above provides one 
answer to this difficult question, attempting to reduce an extremely varied, complex and large- 
scale set of practices to something small and simple, but of value to classroom practice in a 
variety of local contexts. 

Nonetheless, there are significant general problems associated with teaching students about 
scientific writing and argument. Among these, one that looms large is situativity. As I have 
shown above, many of the forms which we present to students for reporting their work are 
derivative from ones invented by and for participants in a radically different community of 
discourse/practice (professional science), in response the unique rhetorical problems that recur 
there. We cannot expect to simply drop these forms into the classroom and have them fit the 
native activity there. 

In fact, they often do not fit. As an example, below is a "Method" section from a research 
report produced by a student in a project-based high school science classroom in the 1994/95 
school year. This project, on smog, was somewhat unique among high-school projects in that the 
student was given several weeks to complete it and was required, in that time, to come up with 
an original question that could be addressed with numerical data. The following excerpt is fairly 
indicative of the misappropriation of scientific genres that I have seen in this setting: 

Method: My problem with this topic was that all I found was the temperature and precipitation data . I sat at 

Mosaic and Netscape for hours just cruising through the information endlessly. I even tried Lycos and all of the 
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other searching mechanisms in order to find the rate of photochemical smog. Nobody had it . This time period 
was quite frustrating. Finally, I posted on a newsgroup. For awhile, I did not hear anything, but finally a very 
nice person wrote me back. A man on the California Air Resources Board sent me quite a bit of information. As 
a result, I had to change mv topic . T decided to try and find a correlation between the precipitation and 
temperature and ozone statistics between 1970 and 1979 . That is when I could get down to business, [my 
underlining] 

When I present this quotation in talks I am often asked what I think is wrong with it. Nothing, I 
reply, unless the goal of having the students write it was to teach them what role a Method 
section plays in a research article as a piece of persuasion. The underlined portions of the 
quotation are those that I believe serve the customary function of a Method section. The first two 
underlined parts explain a practical constraint on the investigation: the desired data could not be 
located. The next part explains a strategic response to this problem: the question is changed to 
fit the available data. The balance of the section, however, is more or less an adventure story, 
told by the student to her teacher, about the difficulty of completing his assignment. This story 
makes careful appeals to considerations associated with assessment: the long hours dedicated the 
project, the resulting frustration, and the student's use of all of the resources made available by 
the teacher. What little the student says about the strategic decisions made in her investigation is 
almost lost amid this argument over grades. 

I would argue that this kind of writing, which is centered on the grades students feel they 
deserve rather than knowledge-claims justified by their research, can make the imitation of 
scientific genres in school worse than useless. Having students routinely produce work like this 
puts us at risk of teaching and reinforcing a caricature of scientific practice, rather than revealing 
anything of the social and intellectual texture of scientific practice and the importance of 
particular modes of persuasion within it. 

Students' Rhetorical Situations 

...the situation controls the rhetorical response in the same sense that the question controls the answer and the 
problem controls the solution. Not the rhetor and not persuasive intent, but the situation is the source and 
ground of rhetorical activity.... (Bitzer 1968, p. 6) 

I would like to argue (and below I will present some evidence that) the misappropriation of 
scientific genres I illustrated above stems from the structure of the situations in which we ask 
students to research and write. To illustrate some of the differences between the situations 
presented by school science and professional science, I have included Figures 3 and 4. Figure 3 
is a diagram by Berkenkotter and Huckin (1996, p. 62) which depicts the role of research articles 
produced for publication in the continuing professional practice of the sciences. Figure 4 is my 
own depiction of the place of paper writing in school science. 
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Figure 3: Place of scientific research articles in professional practice 

What I would like to draw your attention to in Figure 3 is the prominent place of articles written 
for publication in the social and professional credit system of the sciences (what Latour and 
Woolgar (1979) call the "cycle of credit"). While scientists are certainly judged by their peers on 
a myriad of both professional and personal criteria, published articles clearly have a privileged 
influence on a researcher's professional fortunes (Myers 1990). The primary output of academic 
research is publications, which play a major role in "mobilizing" research outcomes in the public 
sphere. 

In contrast to the world of professional science, examine Figure 4, which depicts my own 
impression of the place of paper writing in the life of a middle school or high school student. 
Note that in the school environment, there are many more ways to gain credit with the powers 
that be than by writing. A significant portion of a student's grades may actually derive from the 
teacher’s first-hand observations of how he or she behaves in class: attendance, on-task 
behavior, contribution to discussions, cooperation with classmates, and so on. This variety of 
credit mechanisms necessarily makes writing less important for the student than for the adult 
professional, whose stock of credit can go up or down tremendously from writing (or lack of 
writing) alone. 
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Figure 4: Place of writing in school life 

As a result of the differently-structured credit system that surrounds students' writing, they find 
themselves in very different rhetorical situations than scientists. It should not be surprising to us 
that these situations effect how students appropriate genres of scientific writing, and what the 
writing task consequently teaches them about scientific research. The highly formulaic nature of 
the reports that students write for labs cannot help but engender notions of science as an isolated, 
intellectually and morally risk-free affair: 

The lack of student learning in labs. ..is directly related to the lack of thinking it requires. The careful 
procedures, the concern for safety, and the general atmosphere that penalizes mistakes all mitigate against 
questioning, risk-taking, thinking, and learning. It is as though both teachers and students subscribe to a 
mechanistic model of learning which posits that going through certain steps. ..will somehow magically result in 
learning. (Tinker 1993, pp. 236-237) 
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The Problem of Audience 



I began this paper by. arguing that students' encounters with genres of scientific writing, 
particularly those that we ask them to prepare work in, are likely to have a strong influence over 
their understandings of scientific practice. In the previous section I carried this argument further 
by suggesting that the rhetorical situations in which we place students will, in turn, have a strong 
influence on the ways that students appropriate, or misappropriate, genres of scientific writing. 

The belief that stands behind this position is that teaching students about scientific argument 
means considerably more than teaching them about syllogistic logic or about the formal features 
(e.g. heading structure) of genres alone. These ideas are, as Russell (1991) argues, as absurd as 
teaching someone the rules of movement for chess pieces without teaching them the objective of 
the game. Our objective in teaching students about scientific writing should be to provide them 
with tasks and situations in which they can faithfully appropriate the trappings of scientific 
argument to their own persuasive goals. To the greatest practical extent, those situations and 
those goals should be a match for those of adult scientists. 

It is well enough to assert this, of course, but the trouble becomes how to do it? My 
observations and discussions with teachers over the past four years have led me to believe that 
the best way is to provide students with an attentive audience for their work that doesn't read it 
simply to grade it. One way that my research partners and I have found practical to do this has 
been through telementoring (O'Neill, Wagner and Gomez 1996), a process in which adult 
volunteers use e-mail to consult with teams of students on their ongoing project work. In this 
context , my audience argument can be stated as a testable hypothesis. Through telementoring, 
my research partners and I suspected that both the form and substance of students’ written work 
would become more authentic, because: 

• They would have a critical audience that was not reading their work in order to grade it 

• They would have additional guidance on the selection of data resources, their origins and 
the limits of their meaning 

Of course, my audience hypothesis presented a measurement problem. How would I know if my 
research collaborators and I had changed students' rhetorical situations in the ways that we 
wanted? To address this issue, I developed a coding scheme to support the systematic 
comparison of genre appropriation patterns across students' papers. As it happens, one of the 
most closely studied genres of writing at the present time is the scientific research article (see 
Swales 1990, Chap. 7 for an extensive review of literature). This provides a substantial base of 
scholarship on which to build. 

The RA Genre Appropriation Instrument 

The current version of my coding scheme represents a synthesis of observations about the 
Research Article genre from literature in the sociology of science (Bazerman 1988, Myers 1990) 
and Linguistics (Swales 1990) with observations made through close reading of a corpus of 
roughly 150 research articles written by middle and high school students in two project science 
classrooms. One of these classrooms is in an inner-city setting, the other in an affluent suburban 
setting. 

In its current form, my scheme codes for customary rhetorical moves in the sections of IMRD 
(Introduction, Method, Results, Discussion) Research Articles and their placement, as well as 
many of the other trappings of argument in this genre, such as: 

• The text types employed (narration, expository prose, overt persuasion) 

• Audience and persona indicators (who appears to be writing, and who they appear to be 
writing to) 
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• Ways in which sources and authorities are used to reinforce argument (e.g. precise citation 
of supportive research or vague references to straw men) 

• Number and types of sources used (periodicals, encyclopedias, World-Wide Web pages, 
personal communications, etc.) 

• Treatment of perspectives and opinions (e.g. anticipating objections to the research and 
attempting to address them) 

• Use of hedges and qualifications on arguments 

The current version of this instrument is included in Appendix A. It is worth noting that despite 
having taken inspiration from researchers who have studied genres of professional science, the 
development of this coding scheme was largely inductive. That is, no behaviors are included in 
the scheme that I haven't observed in students' writing, or that aren't simple logical complements 
to behaviors I have observed. Thus, I am not unfairly applying the standards of professional 
science to the work of students. 

The Research Setting 

The setting for the research I will report here was a project-based Earth Science class at a 
suburban Chicago high school. The teacher, Rory Wagner, began developing his project-based 
teaching style 4 1/2 years ago, with the goal of helping his students learn more about how 
scientific research is done and reported. He is now at a stage of development in his teaching at 
which students' independent project work takes up three contiguous quarters of the school year. 
(This is quite impressive given that his non-AP class primarily draws students with low 
motivation to study science.) 

In the first academic quarter, Rory lectures to his students about Earth Science in order to 
give them a grounding in the phenomena they might choose to investigate in the remainder of the 
year. This quarter ends with a typical content test. From that point forward, students are 
evaluated largely on their performance on project work, which they conduct in self-selected 
teams, pursuing research agendas of their own formulation. (When asked to describe what 
phenomena his students are permitted to do research on, Rory responds, "Anything that isn't 
living — that's Biology.") Each project lasts for roughly seven weeks, and the only strict 
requirements are that students pose a clear research question which they can address with some 
form of numerical data analysis, and that they submit a quasi-professional research report on 
their work. Along the way, portions of the paper are submitted as "milestones" and assessed: for 
example, their research questions and the data they plan to analyze. 

Of course, one of the most unusual aspects of the students' working conditions is that each 
team is assigned a volunteer telementor, recruited by Mr. Wagner, to advise and assist the 
research through exchanging periodic e-mail. These telementors, many of whom are 
professionals or masters students in the geosciences, have on many occasions been of great help 
to students in locating data sources for their research and/or suggesting manageable lines of 
investigation. They can also provide a critical sounding-board for students' ideas. While they 
may have some past teaching experience, they are not specially trained. Their orientation to 
telementoring, in fact, consists entirely of a series of e-mail messages describing the nature of the 
class the students are enrolled in, the work they are expected to do, and the kinds of help they 
will need. They are aware that students produce written reports of their work, and sometimes see 
these reports, but they are not complicit in our agenda to teach students about genres of scientific 
reporting. 

Telementoring Activity 

While I have collected a great deal of data on the telementoring activity that took place in this 
setting, I will leave extended discussion of it for another paper. For my purposes here it will be 
sufficient to broadly characterize this activity and the relationship it bears to students' 
appropriation of the Research Article genre. 
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Figure 5 illustrates the volume of e-mail that teams of students reported exchanging with 
their telementors in the 1995/96 school year. On average, teams sent or received a little more 
than one message per week over the length of each project. While a few teams sent or received a 
message every day or every other day, most corresponded with their telementors much less 
frequently than this. In fact, a few teams who were assigned a volunteer mentor claimed never to 
have sent him or her a single message. In written surveys and focus groups, these students 
seemed satisfied that they could do a good enough job on their projects by themselves, and 
preferred not to have their lives complicated by the involvement of a remote collaborator. In 
these cases, the teacher usually did not attempt to force discussion to occur. 




MTraffic/Team 

Figure 5: Number of messages per project that student teams reported exchanging 
with telementors over the 1995/1996 school year 



Most teams' e-mail exchanges with their telementors, while friendly, were quite unlike the well- 
known pen-pal relationships in which the corespondents spend considerable time getting to know 
one another and locating common interests. Rather, these discussions grew directly from and 
supported the Students' ongoing project work, which was described to the telementors in advance. 
In the words of one student, the conversations were "straight down to business", relating largely 
to the data sources that students might employ in their research, they questions they might pose, 
and how they would go about answering them. This provided a basis for students to broach 
career or other interests with their telementors, but these discussions did not take center stage. 

Research Design and Results 

I did not wish to disrupt the unique work ongoing in the research setting simply to establish 
artificial conditions to test my audience hypothesis. Instead, I chose to follow the research model 
of design experiments (Hawkins & Collins 1997), which focuses on exploiting the natural 
variation in the activity and outcomes in a setting to explore the relationships between a set of 
interacting variables . Using the genre appropriation instrument, I coded a sample of 22 reports 
written by teams of students for their second and third projects of the 1995/96 school year. In 
line with my audience hypothesis, my goal was to explore the relationships between genre 
appropriation and: 

• The volume of correspondence between the team and its mentor 




D. Kevin O'Neill 



Prepared for AERA 97, Communities of Practice SIG 



12 



How do the papers written by teams toward the left (low message traffic) end of the scale in 
Figure 5 compare with those written by teams toward the right (high traffic) end of the 
scale? 

• Team members' average grades on a typical Earth Science content test 

Are the students who appropriate scientific genres in an authentic manner the same ones 
who perform well on a traditional academic task? 

• The types of research sources the team mentioned using in its paper 

Do the kinds of source materials used by students in their research bear a measurable 
relationship to patterns of genre appropriation? 



The IMRD Customary Functions. Score 

One of the scores I constructed from the coded papers is an indicator of how well the sections of 
the paper fulfill the customary rhetorical functions of the research article. For instance, in an 
IMRD paper one normally expects the Introduction to state the question or problem the authors 
are attempting to address, explain the importance of this problem to the field, summarize the 
research methods employed by the authors, and summarize the results obtained. Likewise, one 
expects the Discussion section to state the conclusions the researchers have drawn in some detail, 
to explain how the data support these conclusions, and to elaborate on the importance of the 
results with reference to the problem framed in the Introduction. (See Appendix A, "Section- 
Specific Rhetorical Functions" for a complete list of the rhetorical functions coded for in the 
scheme.) 

Over the hundreds of years that the IMRD genre has been developing (Bazerman 1988), 
these rhetorical functions have become customary because they help readers to understand and 
digest the writers' knowledge -claims. Thus, I argue, they come more naturally if the rhetorical 
situation the authors see themselves in lends itself to arguments about knowledge-claims rather 
than grades. 




IMRDScore 

Figure 6: Distribution of scores on the IMRD Functions dimension 
for the sample of papers coded 
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Figure 6 shows the distribution of IMRD Elements scores for the sample of 22 student papers I 
will be discussing. Note that the shape of this distribution is roughly normal. Because this 
dimension refers to structural elements in the paper's rhetoric, papers with high scores do not 
necessarily reflect the highest-quality research, but report it in a way that more clearly indicates 
its value, or lack of value, as a piece of scientific research. Papers with very low scores on this 
dimension are typically what one would describe as book reports, whose content has been 
shoehorned into the required sections of the research article. In other words, low-scoring papers 
use some of the formal features of the IMRD genre, but the conduct of the research and the 
content of the paper have not been informed by their implicit messages. 

As it turns out, the volume of correspondence that teams reported sharing with their 
telementors correlated significantly with their IMRD scores (r=.499, p<.02). That is, in this 
sample, correspondence with telementors and authentically-structured papers go together. 
Interestingly, students' survey reports concerning the nature of relationship they had with their 
telementor (for instance, their impressions of the mentor's friendliness or the respect he or she 
demonstrated for the students) do not correlate significantly with the IMRD score, though 
perhaps when more papers are coded this relationship will come into focus. 

It would be natural to suspect that the students whose papers had the most authentic 
rhetorical structure might be the most dedicated or the brightest. While I do not have data to 
address the "brightness" issue, I can report that I found no significant correlation between the 
IMRD scores earned by each team's paper and the average grade received by team members on 
the Earth Science content test administered at the start of the 95/96 school year. Whether one 
considers performance on this test an indicator of students' learning ability or their dedication to 
success in the class, it was not the case that students who performed well on this traditional 
academic task necessarily produced the most authentically-structured rhetoric in their reports. 

Last of all, in light of my argument above concerning the importance of source materials in 
shaping students' genre knowledge, one might expect to find an empirical relationship between 
measures of genre appropriation and the types of research sources students used in their work. 
Surprisingly, I did not find any significant correlation between the IMRD scores of my sample of 
reports and the particular types of research sources (e.g. encyclopedias, popular press articles, 
web pages) students mentioned having used in the preparation of those reports. So within the 
scope of this (admittedly small) dataset there was no clear suggestion of the effects of research 
sources on students' genre appropriation. 

The Perspectives and Opinions Score 

Another score that I constructed for each team's research article is intended to draw together 
evidence of even-handed argument. This dimension distinguishes between papers that simply 
make bald assertions about the research findings and those that make careful qualifications or 
strategically give ground to strengthen their arguments. Students earn points on the perspectives 
and opinions dimension by: 

• Anticipating possible objections and attempting to address them (e.g. admitting that 
sampling bias may be an issue in the research design and arguing that the sampling strategy 
used was adequate) 

• Acknowledging the viability of alternate interpretations (e.g. admitting other possible 
interpretations of the data left by measurement error) 

• Summarizing and weighing the merits of more than two theories or interpretations that 
explain the data 

• Drawing attention to a point of general disagreement in the field (e.g. weaknesses in data 
sources) 

The score awarded to the paper, and the group of students who authored it, is the total number of 
occurrences of all of these features. Therefore, a team can earn two points by anticipating two 
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possible objections to their research methods or data analysis, and so on with each of the 
components of this score. 




POScore 

Figure 7: Distribution of scores on the Perspectives and Opinions dimension 
for the sample of papers coded 

As the skewed distribution of scores in Figure 7 illustrates, it was relatively rare for the student 
research teams to display this kind of even-handedness in their written reports. However, when 
they did, they were very likely also to have reported a high volume of correspondence with their 
telementors (r=.759, p<.0001). While correlation does not imply causation, it is reasonable to 
suspect that the strength of the relationship between these variables is a result of students' 
conversations with their telementors, in which they became acquainted with the flaws in their 
work and the range of possible objections to it. A careful analysis of the e-mail exchanges 
between the student teams and their telementors will be conducted in the near future to 
investigate this hypothesis, and to determine if particular strategies on the part of telementors 
might have led to these positive outcomes. 

Finally, it should be noted that as with the IMRD score above, there was no significant 
correlation between the team's P/O score and its average grade on the first-quarter content test. 
Therefore, it is more reasonable to attribute the students' even-handed arguments to their 
experience of telementoring than to their previously-demonstrated studiousness or aptitude in the 
content area. 

Summary/Conclusions 

I opened this paper by arguing that the development of students' understanding of genres of 
scientific writing is an important and long-traditional objective in science education. Not only do 
genre conventions represent a common way of organizing and assessing individual and group 
activity in science classrooms, but they are a useful vehicle for developing students' 
understanding of communities of practice in science. Unfortunately, genre theory and the 
research reported here suggest that the ways in which students appropriate scientific genres in the 
classroom are strongly dependent on the situations in which we ask them to write. In common 
teaching and learning practice, the lack of an audience for students' work other than their teacher 
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puts them at risk of misappropriating scientific genres and learning crude caricatures of scientific 
practice. 

The latter part of this paper presented an argument and some preliminary research findings to 
suggest that by involving new audiences in students' work, such as volunteer telementors, it is 
possible to change the "rhetorical situations" in which we ask students to emulate scientific 
genres of research and writing, so that they can more faithfully appropriate these genres. 

Through the use of coding instruments such as the one discussed here, it is also possible to 
measure the influence of novel teaching techniques on students' genre appropriation. Such 
measures, if duly validated and tested on larger corpora of students' work, could serve a crucial 
role in the evaluation and re-design of interventions aimed at developing students' 
understandings of communities of scientific discourse and practice. 
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Appendix A: Student Science RA Rhetoric Coding 

Paper Title: 

Team Number: 

Date Submitted: : , 

Writers & IDs 1 ID 

ID 

ID 

ID 

• ID 

ID 

Teacher, Period: 

Date Coded: Coder: 



Notes: 





Section-Specific Rhetorical Functions 2 



RA Sections Present (i.e. marked with a labeled heading) 



□ Abstract 


□ Introduction 
(sometimes labeled 
’'Background") 


□ Method 
(sometimes labeled 
"Procedure") 


□ Results 
(sometimes labeled 
"Data”) 


□ Discussion ! 

(sometimes labeled 
"Conclusions") 


□ References 
(sometimes labeled 
“Bibliography” or 
“Literature Cited”) 


Other Sections Present 


□ Materials 


□ Purpose 


□ Acknowledgments 







Abstract/Introduction 





□ States a purpose in the form 
of a problem, question, or issue 
to be resolved 

In this section? 

□ Y □ N 


□ Explains the significance of 
this purpose to the general 
audience (i.e. why do we care?) 

In this section? 

□ Y □ N 


□ Summarizes Method 

In this section? 

□ Y □ N 


□ Summarizes Results 

In this section? 

□ Y □ N 


□ Provides background 


□ Summarizes important 






research into the broad topic 


findings from earlier work on 






area 


the problem by others (names 








names and gives citations) 






In this section? 


In this section? 






□ Y □ N 


□ Y □ N 







Methods 



□ Describes what was done by 
the investigators (built a 
physical model, collected 
samples, gathered research 
sources from libraries or 
electronic archives, etc.) 


□ Does this description in 
terms and with precision 
appropriate to others who 
might want to reproduce the 
results 


□ Explains why the method 
followed by the writers could 
be expected to lead to a 
resolution of the problem, or to 
an answer to the question 


□ Mentions specific search 
goals and/or criteria employed 
at library or in Internet searches 
(e.g. ‘we searched with the 
keywords "sea surface 
temperature’") 


In this section? 

□ Y □ N 


In this section? 

□ Y □ N 


In this section? 

□ yon 


In this section? 

□ YON 



Results 



□ Foreshadows results briefly 


□ Presents the data collected or 


□ Characterizes or "glosses" 


□ Provides an interpretation of 


(in a sentence or two) 


found (in tabular or graphical 
form, a set of images, etc.) 


the data for the non specialist 


data with respect to the original 
question (in the form of 
calculations and/or prose that 
refers specifically to the data) 


In this section? 


In this section? 


In this section? 


In this section? 


□ yon 


□ yon 


□ yon 


□ yon 



Discussion 



□ States the conclusions that 


□ Attempts to explain how the 


□ Discusses the importance of 


□ Makes suggestions for 


can be made about the original 
question or problem from the 
data or information collected 


data support, refute, or are 
unrelated to a question or 
problem mentioned earlier 


the results with reference to the 
significance of the original 
question 


further study 


In this section? 


In this section? 


In this section? 


In this section? 


□ yon 


□ yon 


□ YON 


□ YON 
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Persuasive Features 
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Text Types Used (check at least one ) 



□ Expository (lays out facts, frameworks, 
or key terms) 



□ Narration (relates a sequence of events 
in past tense) 



□ Overt Persuasion (argues for a 
particular interpretation with words like 
“since. ..then...”, “therefore”, or “because”) 



Persona Indicators - how do the writers present themselves? (check all that apply) 



□ Describes scientists as Other (e.g. “Scientists say...”) 


□ Assumes guise of scientist (e.g. “In our professional 
opinion...”) 


□ Speaks in first person (uses “I”, “We”, “Our”) 


□ Uses passive voice to exclude self (e.g. “A data table was 
constructed” rather than “We constructed a data table”) 



Audience Indicators - who does this seem to be writ! 


ten to? (record line numbers and count) 


B 

□ Refers to teacher in 3rd 
person, or not at all 


B 

□ Addresses teacher by 
name (e.g. “Well Mrs. 
Whitcomb, what we decided to 
do was...”) 


B 

□ Refers to experience 
shared with teacher or assumes 
knowledge that only the 
teacher or other classmate(s) 
would have 


B 

□ Uses colloquialisms 
common to students but not to 
adults 



Use of Sources and Authorities (record line numbers and count) 



Cites named sources for 
support (e.g. "As (Myers 1985) 
has said...") 



Cites named sources as 
examples of opposing views 
(e.g. "Counter to findings by 
Wallin (1995), our research 
showed...") 



Provides precise machine 
addresses, paths and filenames 
for Internet resources 



Identifies originators of 
data or information used by 
name of person or institution , 
or title of work (not just by the 
media through which they were 
obtained, e.g. World Wide 
Web) 



Uses unnamed authorities 
for support (e.g. “Many 
scientists believe...”) 



Alludes to unnamed 
opponents, perhaps as straw 
men (e.g. “many people 
think.. .but really...”) 



Makes vague references to 
books or Internet sources (e.g. 
“The following data come from 
a Mosaic page” or “This data 
came from a weather book”) 



Acknowledges the 
contributions of non-team- 
members to the work presented 
(e.g. other student, mentor, 
teacher, or other adult) 



Types of Sources and Authorities t 


Mentioned (check) - (these do not need to be bibliographical refs.) 


□ Textbooks 


□ Encyclopedias 


□ Popular Press 


□ Internet 
resources (e.g. web 
pages) 


□ Personal 
Communications 


□ Scientific 
Journals 



Treatment of Perspectives and Opinions (record line numbers and count) 



B 

Anticipates possible 
objections and attempts to 
address them (e.g. “It could be 
said that.. .but on the other 
hand...”) 


B 

Summarizes and weighs 
several perspectives on a point 
(i.e. evaluates more than two 
possibilities for their relative 
merit) 


B 

Acknowledges viability of 
a view(s) not shared by the 
writers (e.g. “It could still be 
the case that...”) 


B 

Draws attention to a point 
of general contention in the 
field (e.g. “There seems to be 
little consensus over how to 
interpret the ice core data...”) 


# 


# 


# 


# 
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Hedging (record line numbers and count) 



a 

Acknowledges possible flaws in 
method or calculations performed by the 
writers 


B 

Acknowledges the limits of the 
writer’s own experience or data 


B 

Emphasizes the provisional nature of 
the conclusion or argument put forward 
(e.g. “More data are needed for a definite 
answer to this question”) 




# 




# 




# 








Overall Reflections (record line numbers and count) 


Proclaims success 


Proclaims lesson learned about the nature 
or practice of science 


Proclaims that important science content 
was learned. 




# 




# 




# 








Proclaims failure 


Proclaims lesson learned about doing 
investigations 


Proclaims lesson learned about working in 
groups 




# 




# 




# 








Complaints, Excuses, etc. (record 1 


line numbers and count) 


Claims crucial resources were not 
available or convenient enough 


Claims team conflict impeded progress 


Claims the problem looked deceptively 
easy, but later proved hard 




# 




# 




# 








Claims teacher was unhelpful 


Claims mentor let the team down 


Talks about long hours or hard work 
invested in the project 




# 




# 




# 








Claims there wasn’t sufficient time to do 
the assignment 


Claims the assignment was hard to 
understand 






# 




# 
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